Reduce russh write-path copies with direct Bytes sends by mjc · Pull Request #695 · Eugeny/russh

mjc · 2026-04-23T23:07:21Z

Summary

This PR reduces write-path copying in russh by moving more packet assembly directly into PacketWriter and by sending owned channel payloads as Bytes where ordering allows.

For channel data, the new fast path writes Bytes payloads straight into PacketWriter instead of first staging plaintext into Encrypted.write. That path is used only when the channel is confirmed, no rekey is in progress, there is no older pending data for that channel, and the session write queue is empty. If any of those conditions are not met, behavior falls back to the existing queued path.

The same direct-writer path is now used when replaying pending channel data after a window adjustment or after rekey completion, again only when ordering permits. This keeps the existing ordering guarantees while avoiding the extra staging copy on the replay path too.

API additions

This also adds owned-Bytes channel send helpers:

Channel::data_bytes
Channel::extended_data_bytes
ChannelWriteHalf::data_bytes
ChannelWriteHalf::extended_data_bytes

These helpers split payloads by writable window and max packet size using Bytes::slice(...), so callers that already own a Bytes buffer can enqueue channel data without the extra copy inherent in the AsyncWrite<&[u8]> path.

Packet writer changes

To support the direct path, PacketWriter is refactored so it can assemble and seal packets directly into its output buffer when compression is disabled, while still keeping a reusable plaintext packet buffer for retained-packet and compressed cases. The compressed path is also tightened so compressed output can be written directly into the final output buffer.

Retained KEXINIT-style packets are now kept as Bytes, and the stored client/server KEXINIT payloads in Exchange use Bytes as well.

Hardening and validation

This PR also hardens channel write sizing by returning Error::Inconsistent when a peer advertises a zero max packet size instead of panicking.

Coverage was expanded around:

direct channel-data writes
pending-data replay through PacketWriter
rekey replay behavior
retained packet handling
zlib + max-channel-packet-size behavior

Wall-Clock Measurements

Temporary harnesses, not committed. Base is updated main at c31cbc9.

Numbers below are median-of-5 runs from the same local harness.

case	main path	branch path	main median ns/op	branch median ns/op	faster by
channel data, 32 KiB	current `Encrypted.write` staging	direct `Bytes` -> `PacketWriter`	1,038.75	649.63	37.5%
channel data, 256 KiB	current `Encrypted.write` staging	direct `Bytes` -> `PacketWriter`	10,418.58	5,690.90	45.4%
pending replay, 32 KiB	replay into `Encrypted.write`	replay into `PacketWriter`	945.84	520.06	45.0%
pending replay, 256 KiB	replay into `Encrypted.write`	replay into `PacketWriter`	9,649.69	4,508.97	53.3%
channel API, 32 KiB	`AsyncWrite<&[u8]>` copy path	owned `Bytes` API	657.25	135.53	79.4%
channel API, 256 KiB	`AsyncWrite<&[u8]>` copy path	owned `Bytes` API	6,256.34	953.55	84.8%

IAI/Callgrind Measurements

Same temporary harness. Instruction counts:

case	main Ir	branch Ir	reduction
channel data, 32 KiB	82,811	48,104	41.9%
channel data, 256 KiB	678,528	280,143	58.7%
pending replay, 32 KiB	83,414	48,609	41.7%
pending replay, 256 KiB	679,144	280,730	58.7%

Retained packet construction is effectively flat by IAI: 67,817 Ir on main vs 67,931 Ir here for the 32 KiB retained packet case.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR optimizes russh’s outbound write path by enabling a direct Bytes -> PacketWriter -> encrypted output fast path when ordering constraints allow, reducing intermediate plaintext staging/copies while preserving packet ordering and encryption behavior.

Changes:

Introduces PacketWriter::write_packet in-place encryption (plus supporting APIs) and routes more writes through PacketWriter directly when safe.
Adds direct/with-writer channel data flush paths (including replay after window adjust / rekey completion) and new Channel::*_bytes APIs for copy-free Bytes callers.
Hardens handling of peer-provided zero recipient_maximum_packet_size to return an error (instead of panicking / misbehaving), and expands test coverage (incl. optional zlib).

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
`russh/tests/test_max_channel_packet_size.rs`	Extends max-channel-packet-size regression test to cover optional zlib compression.
`russh/src/sshbuffer.rs`	Adds in-place `PacketWriter::write_packet`, retained `Bytes` packet APIs, and related unit tests.
`russh/src/session.rs`	Adds direct channel-data write/flush paths via `PacketWriter` and switches retained KEXINIT storage to `Bytes`.
`russh/src/server/session.rs`	Routes server session channel writes through the new `Encrypted::*_with_writer` fast path.
`russh/src/server/mod.rs`	Adjusts rekey flow to apply new keys before replaying pending channel data through `PacketWriter`.
`russh/src/server/kex.rs`	Updates packet emission to use `write_packet`; stores incoming KEXINIT as `Bytes`.
`russh/src/server/encrypted.rs`	Replays pending channel data through `PacketWriter` on window adjust to avoid staging copies.
`russh/src/negotiation.rs`	Emits KEXINIT via `packet_bytes` and returns retained KEXINIT as `Bytes`.
`russh/src/lib_inner.rs`	Removes an unused `debug` import (minor cleanup).
`russh/src/kex/hybrid_mlkem.rs`	Updates tests to the new `Exchange { *_kex_init: Bytes }` type.
`russh/src/compression.rs`	Adds `Compress::compress_into` to write compressed payloads directly into an output buffer.
`russh/src/client/session.rs`	Routes client session channel writes through the new `Encrypted::*_with_writer` fast path.
`russh/src/client/mod.rs`	Adjusts rekey flow to apply new keys before replaying pending channel data through `PacketWriter`.
`russh/src/client/kex.rs`	Updates packet emission to use `write_packet`; stores incoming KEXINIT as `Bytes`.
`russh/src/client/encrypted.rs`	Replays pending channel data through `PacketWriter` on window adjust; updates one packet emission call site.
`russh/src/cipher/mod.rs`	Introduces `SealingKey::finish_packet` to support in-place packet finalization/sealing.
`russh/src/channels/mod.rs`	Adds `data_bytes` / `extended_data_bytes` APIs to send owned `Bytes` without copying into the `AsyncWrite` path, with tests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mjc · 2026-04-27T19:43:40Z

if desired I can add the benchmark harness, also. just wasn't sure it would be wanted

Eugeny · 2026-04-28T06:46:02Z

Thank you so much! It will take me a while to properly review since I'm currently away until the 1st - but from a cursory glance everything looks good 👍

mjc · 2026-04-28T15:41:39Z

Yeah, sorry about the size of it. if it helps I could split it into a bunch of smaller PRs for easier examination when you're back.

There are a couple ways to slice this particular pie that might make it simpler.

Eugeny · 2026-05-16T21:04:54Z

Thanks a ton, great work 👍

mjc and others added 10 commits April 22, 2026 22:33

refactor: reuse plaintext packet buffers

d601a16

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

test: cover retained packet buffer reuse

260c9a1

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

refactor: keep reusable packet buffer for retained packets

5aad309

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

test: add follow-up write-path coverage

3640780

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Refine compressed write path

77d832f

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Clarify write_packet buffer contract

ff662c6

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

refactor: move retained packets into Bytes

86d2e42

refactor: write channel data directly from Bytes

0579c8c

refactor: replay pending channel data through PacketWriter

7a03b8b

api: add Bytes channel data helpers

5632339

Copilot AI review requested due to automatic review settings April 23, 2026 23:07

Copilot started reviewing on behalf of mjc April 23, 2026 23:07 View session

mjc added 5 commits April 23, 2026 17:14

fix: tighten PacketWriter fast path

45ec0ec

fix: harden direct channel write path

73255f3

fix: harden Bytes channel send path

0edb78b

fix: switch pending replay to new keys after rekey

d37c5d7

fix: keep server write window in sync after replay

0bfd1ac

mjc changed the title ~~Refine russh write path~~ Reduce russh write-path copies Apr 23, 2026

Copilot AI reviewed Apr 23, 2026

View reviewed changes

mjc force-pushed the write-path-refactor-main branch from 1f3a306 to 0bfd1ac Compare April 23, 2026 23:15

mjc changed the title ~~Reduce russh write-path copies~~ Reduce russh write-path copies with direct Bytes sends Apr 23, 2026

Eugeny merged commit 32fd46f into Eugeny:main May 16, 2026
11 checks passed

mjc deleted the write-path-refactor-main branch May 16, 2026 22:48

Navigation Menu

Uh oh!

Reduce russh write-path copies with direct Bytes sends - #695

Reduce russh write-path copies with direct Bytes sends#695
Eugeny merged 15 commits into
Eugeny:mainfrom
mjc:write-path-refactor-main

mjc commented Apr 23, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

mjc commented Apr 27, 2026

Uh oh!

Eugeny commented Apr 28, 2026

Uh oh!

mjc commented Apr 28, 2026

Uh oh!

Eugeny commented May 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

mjc commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

API additions

Packet writer changes

Hardening and validation

Wall-Clock Measurements

IAI/Callgrind Measurements

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

mjc commented Apr 27, 2026

Uh oh!

Eugeny commented Apr 28, 2026

Uh oh!

mjc commented Apr 28, 2026

Uh oh!

Eugeny commented May 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mjc commented Apr 23, 2026 •

edited

Loading